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REMARKS 

This Amendment is in response to the Office Action dated February 
27, 2006 ("OA"). In the Office Action, claims 1-28 were rejected under 
35 USC §103. By this Amendment, claims 1, 16 and 22 are amended. 
Currently pending claims 1-28 are believed allowable, with claims 1, 16 
and 22 being independent claims. 

CLAIM REJECTIONS UNDER 35 USC §103 

Claims 1-28 were rejected under 35 USC .§103 as obvious over U.S. 
Patent No. 6,804,684 issued to Stubler et al . (hereinafter "Stubler") 
in view of U.S. Patent No. 6,816,847 to Toyama (hereinafter "Toyama") . 
OA, pg. 2. 

A prima facie case for obviousness can only be made if the 
combined reference documents teach or suggest all the claim 
limitations. MPEP 2143. Furthermore, to establish a prima facie case 
of obviousness, there must be some suggestion or motivation to modify 
the reference or to combine reference teachings. MPEP 2143. 

Before addressing the subject matter of the claims, a general 
discussion of the present invention, Stubler and Toyama may assist 
review of the pending application. 

The present invention relates to efficient interactive annotation 
or labeling of multimedia content to facilitate effective searching, 
filtering and usage of content. App., pg. 1, In. 4-6. In one 
embodiment of the invention, efficiency is realized by prompting a user 
to annotate only a small set of selected example content, with the 
results propagated to the annotation of a maximum number of other 
multimedia content. App., pg . 4, In. 11-14, pg . 7, In. 2-4. This 
causes the user to annotate as few examples as possible. App., pg . 5, 
In. 7-8. 

Stubler appears to relate to a method for associating captions 
with images in an image database environment. Stubler, col. 1, In. 7- 
10. Briefly, the captioning technique of Stubler involves acquiring an 
image for evaluation with respect to stored images, automatically 
extracting metadata from the acquired image, automatically selecting 
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one or more stored images having metadata similar to the extracted 
metadata, and generating one or more captions for the acquired image 
from preexisting captions associated with the selected stored images. 
Stubler, col. 3, In. 15-32. Although Stubler discusses an optional 
feature for user verification of extended captioning or semantic 
labeling, there is no teaching of actively selecting examples of 
multimedia content to achieve the annotation of a maximum number of 
other multimedia content. Stubler, col. 8, In. 40-55. 

Toyama appears to relate to computerized aesthetic judgment of 
images. Toyama, col. 1, In. 47-48. According to Toyama, a classifier 
is trained based on a set of training images, along with aesthetic 
scores assigned to the training images. Toyama, col . 5, In. 15-19. 
The training images can be scored by professionals and/or laypeople. 
Toyama, col. 5, In. 32-38. Once trained, the classifier can be used to 
provide aesthetic scores to input images, as well as provide 
recommendations to improve the aesthetic scores of the input images. 
Toyama, col. 1, In. 65 - col. 2, In. 4. 

Claim 1 

Claim 1 is amended here in to recite, in part, "actively selecting 
examples of multimedia content to be annotated by a user, wherein the 
examples of multimedia content are selected based on at least one 
criterion for achieving a maximal disambiguation result , such that only 
those examples which are most ambiguous are selected ." No new matter 
is believed added by this amendment, and support for the subject matter 
introduced can be found at least at page 7, lines 2-4 of the 
Appl i cat ion . 

In rejecting claim 1, the Office Action argues that although 
Stubler does not teach actively selecting examples of multimedia 
content to be annotated by a user, wherein the examples of multimedia 
content are selected based on at least one criterion for achieving a 
maximal disambiguation result, this teaching is found in Toyama. The 
Office Action cites column 5, lines 15-46 of Toyama in support of its 
posit ion . 

The cited passages of Toyama state, 
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In this section of the detailed description, a method for training 
a classifier, according to one embodiment of the invention, is 
described. Referring to FIG. 2, a flowchart of the method is 
shown. In 200, a training set of images and corresponding 
aesthetic scores for the images is input. The images are desirably 
the same type of images that are to be later judged. For example, 
the images may include a set of web pages, a set of scanned-in 
pictures, a set of created pictures, a set of drawings, a set of 
page layouts, etc. The aesthetic scores for the images are 
desirably made by graphics professionals. Thus, the web pages are 
desirably scored by professional web page designers, the scanned- 
in pictures by professional photographers, the created pictures by 
professional artists, the drawings by professional drawers, etc. 

The set of images in the training set desirably includes a wide 
variety of images, both those considered aesthetically pleasing, 
and those considered aesthetically poor . Likewise, the aesthetic 
scores for each image desirably includes a number of such scores, 
by a diverse number of professionals or laypeople, groups of which 
may be intentionally selected for their common taste (e.g., people 
who prefer The New York Times to the Wall Street Journal, people 
who read Wired Magazine, etc.). Each image is desirably scored 
manually by each professional (or each person surveyed) by 
whatever criteria the person wishes to use for deeming the 
aesthetics of the image, or according to some standard specified 
by the survey. Each image may, for example, be scored on a number 
basis, such as from zero to one-hundred, or, for example, on a 
classification basis, where there are a number of categories, such 
as "excellent," "good," "average," "poor," etc. Toyama , col. 5, 
In. 15-46. 

The Examiner takes the position that Toyama shows that selected 
examples are annotated by a user and wherein the examples provide for 
achieving a maximal disambiguation result. OA, pg . 8. The Applicants 
respectfully disagree with the Examiner. 

Claim 1 recites that the examples of multimedia content are 
selected based on at least one criterion for achieving a maximal 
disambiguation result, such that only those examples which are most 
ambiguous are selected. Toyama contains no such teaching. The only 
selection criterion discussed in Toyama is whether the training set is 
aesthetically pleasing or aesthetically poor. Thus, the Applicants 
respectfully submit that Toyama does not provide any instruction for 
selecting images based on at least one criterion for achieving a 
maximal disambiguation result, as recited in claim 1 of the present 
invention . 
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In rejecting claims under U.S.C. §103, the Examiner bears the 
initial burden of establishing a prima facie case. In re Oetiker , 977 
F.2d 1443, 1445, 24 USPQ 1443, 1444 (Fed. Cir. 1992). To establish 
prima facie obviousness of a claimed invention, all the claim 
limitations must be taught or suggested by the prior art. In re Royka , 
490 F.2d 981, 180 USPQ 580 (CCPA 1974). "All words in a claim must be 
considered in judging the patentability of that claim against the prior 
art." In re Wilson , 424 F.2d 1382, 1385, 165 USPQ 4 94, 496 (CCPA 
1970) . 

The Office Action also argues, "It would have been obvious and 
desirable to have used the active selection techniques disclosed by 
Toyama in col. 5 lines 15-46 to have improved the active selection 
techniques disclosed by Toyama." The Applicants respectfully submit 
that Toyama makes no mention of an active selection technique. 
Specifically, Stubler teaches generating captions or labels for 
acquired images from preexisting captions or labels associated with 
stored images. There in no mention in Stubler of an active selection 
technique . 

The Office Action alleges that the field of endeavor is very 
similar for Stubler and Toyama since Toyama "is directed towards 
propagating annotations from a known training set to similar images 
that have do not have any associated annotations." The Applicant 
respectfully submits that Toyama does not mention or suggest 
propagating annotations from a known training set to similar images 
that do not have any associated annotations. The citation offer by the 
Examiner (col. 1, line 47 to col. 2, line 14 of Toyama) supporting the 
Office Action's assertion is devoid of teaching relating to 
annotations . 

Furthermore, the Office Action fails to cite any suggestion or 
motivation in the prior art to modify the Stubler with the teachings of 
Toyama. Toyama discusses techniques for training a classifier to 
provide aesthetic scores for input images. Stubler does not 
demonstrate any appreciation of training techniques such as discussed 
in Toyama. Thus, the Applicant respectfully submits that no motivation 
or suggestion exists in the cited art to modify the Stubler with the 
teachings of Toyama. 
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For at least the reasons discussed above, the Applicant 
respectfully asserts that the Office Action has not established a prima 
facie case of obviousness for claim 1. As such, the rejection of claim 
1 should be withdrawn. Moreover, the Applicant believes that claim 1 
is patentable over the cited documents and earnestly requests an 
indication of allowability for claim 1. 

Claims 2-15 

Claims 2-15 are dependent on and further limit claim 1. Since 
claim 1 is believed allowable, claims 2-15 are also believed allowable 
for at least the same reasons as claim 1 . 

Claim 23 

Claim 23 recites, "The method of claim 1, wherein the at least one 
criterion includes an ambiguity level of the selected examples." In 
rejecting claim 23, the Office Action argues that although Stubler does 
not teach wherein at least one criterion include an ambiguity level of 
selected examples, such a teaching is found at column 5, line 15 to 
column 6, line 50 of Toyama . The Applicant respectfully disagrees with 
the Examiner's interpretation of Toyama. 

The Applicants find only two sentences that deal with selecting 
examples in the citation provided by the Examiner. The first sentence 
states, "For example, the images may include a set of web pages, a set 
of scanned-in pictures, a set of created pictures, a set of drawings, a 
set of page layouts, etc." The second sentence states, "The set of 
images in the training set desirably includes a wide variety of images, 
both those considered aesthetically pleasing, and those considered 
aesthetically poor." Neither sentence mentions or suggests wherein the 
at least one criterion includes an ambiguity level of the selected 
examples . 

The Examiner also argues that the statement, "Each image may, for 
example, be scored on a number basis, such as from zero to one-hundred, 
or, for example, on a classification basis, where there are a number of 
categories, such as ^excellent,' 'good, ' ^average,' x poor, ' etc." found 
at column 5, lines 42-46 of Toyama teaches wherein at least one 
criterion include an ambiguity level of selected examples. OA, pg . 9. 
The Applicants, however, submit that the above-citation relates to 
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aesthetic categories of images, not ambiguity levels of selected 
examples . 

The Examiner further urges that Toyama teaches an ambiguity level 
of the selected examples at column 6, lines 19-31. This citation 
offered by the Examiner states, 

For example, Support Vector Machines build classifiers by 
identifying a hyperplane that separates a set of positive and 
negative examples with a maximum margin. In the linear form of SVM 
that is employed in one embodiment, the margin is defined by the 
distance of the hyperplane to the nearest positive and negative 
cases for each class. Maximizing the margin can be express as an 
optimization problem and search and optimization thus lay at the 
core of different SVM-based training methods. A post -process ing 
procedure described in the Piatt reference is used that employs 
regularized maximum likelihood fitting to produce estimations of 
posterior probabilities. The method fits a sigmoid to the score 
that is output by the SVM classifier. Toyama, col. 6, In. 19-31. 

The Applicants respectfully submit that this passage relates to 
using Support Vector Machines (see 

http : / /en . wikipedia . org/ wiki /Support _vector_machines attached herewith) 
for creating functions from a set of labeled training data. The 
passage contains no teaching or suggestion of using ambiguity levels of 
selected examples. 

For at least these reasons, and the reasons given for claim 1, 
claim 23 is believed allowable. The Applicant therefore earnestly 
requests an indication of allowability for claim 23. 

Claim 24 

Claim 24 recites, "The method of claim 1, wherein the at least one 
criterion includes a confidence level of the selected examples, the 
confidence level being inversely proportional to a distance of a new 
feature of the selected examples from a separating hyperplane in an 
induced higher dimensional feature space." In rejecting claim 24, the 
Office Action argues that although Stubler does not teach wherein the 
at least one criterion includes a confidence level of the selected 
examples, the confidence level being inversely proportional to a 
distance of a new feature of the selected examples from a separating 
hyperplane in an induced higher dimensional feature space, such a 
teaching is found at column 5, line 15 to column 6, line 50 of Toyama. 
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The Applicant respectfully disagrees with the Examiner's interpretation 
of Toyama . 

The Applicants find only two sentences that deal with selecting 
examples in the citation provided by the Examiner. The first sentence 
states, "For example, the images may include a set of web pages, a set 
of scanned-in pictures, a set of created pictures, a set of drawings, a 
set of page layouts, etc." The second sentence states, w The set of 
images in the training set desirably includes a wide variety of images, 
both those considered aesthetically pleasing, and those considered 
aesthetically poor." Neither sentence mentions or suggests wherein the 
at least one criterion includes a confidence level of the selected 
examples, the confidence level being inversely proportional to a 
distance of a new feature of the selected examples from a separating 
hyperplane in an induced higher dimensional feature space. 

The Examiner also argues that the statement, "Each image may, for 
example, be scored on a number basis, such as from zero to one-hundred, 
or, for example, on a classification basis, where there are a number of 
categories, such as 'excellent, ' 'good,' 'average,' 'poor,' etc." found 
at column 5, lines 42-46 of Toyama teaches wherein the at least one 
criterion includes a confidence level of the selected examples, the 
confidence level being inversely proportional to a distance of a new 
feature of the selected examples from a separating hyperplane in an 
induced higher dimensional feature space. OA, pg . 9. The Applicants, 
however, submit that the above-citation relates to aesthetic categories 
of images, not a confidence level inversely proportional to a distance 
of a new feature of the selected examples from a separating hyperplane 
in an induced higher dimensional feature space. 

The Examiner further urges that Toyama teaches a confidence level 
inversely proportional to a distance of a new feature of the selected 
examples from a separating hyperplane in an induced higher dimensional 
feature space at column 6, lines 19-31. This citation offered by the 
Examiner states, 

For example, Support Vector Machines build classifiers by 
identifying a hyperplane that separates a set of positive and 
negative examples with a maximum margin. In the linear form of SVM 
that is employed in one embodiment, the margin is defined by the 
distance of the hyperplane to the nearest positive and negative 
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cases for each class. Maximizing the margin can be express as an 
optimization problem and search and optimization thus lay at the 
core of different SVM-based training methods. A post -processing 
procedure described in the Piatt reference is used that employs 
regularized maximum likelihood fitting to produce estimations of 
posterior probabilities. The method fits a sigmoid to the score 
that is output by the SVM classifier. Toyama , col. 6, In. 19-31. 

The Applicants respectfully submit that this passage relates to 
using Support Vector Machines (see 

http : //en . wixipedia.org/wiki/Support_vector_machines attached herewith) 
for creating functions from a set of labeled training data. The 
passage contains no teaching or suggestion of using a confidence level 
inversely proportional to a distance of a new feature of the selected 
examples from a separating hyperplane in an induced higher dimensional 
feature space. 

For at least these reasons, and the reasons given for claim 1, 
claim 24 is believed allowable. The Applicant therefore earnestly 
requests an indication of allowability for claim 24. 

Claim 16 

Claim 16 recites, in part, "wherein the examples of multimedia 
content are selected based on at least one criterion for achieving a 
maximal disambiguation result such that only those examples which are 
most ambiguous are selected . " 

In rejecting claim 16, the Office Action argues that although 
Stubler does not teach actively selecting examples of multimedia 
content to be annotated by a user, wherein the examples of multimedia 
content are selected based on at least one criterion for achieving a 
maximal disambiguation result, this teaching is found in Toyama. The 
Office Action cites column 5, lines 15-46 of Toyama in support of its 
position . 

The cited passages of Toyama state, 

In this section of the detailed description, a method for training 
a classifier, according to one embodiment of the invention, is 
described. Referring to FIG. 2, a flowchart of the method is 
shown. In 200, a training set of images and corresponding 
aesthetic scores for the images is input. The images are desirably 
the same type of images that are to be later judged. For example, 
the images may include a set of web pages, a set of scanned-in 
pictures, a set of created pictures, a set of drawings, a set of 
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page layouts, etc. The aesthetic scores for the images are 
desirably made by graphics professionals. Thus, the web pages are 
desirably scored by professional web page designers, the scanned- 
in pictures by professional photographers, the created pictures by 
professional artists, the drawings by professional drawers, etc. 

The set of images in the training set desirably includes a wide 
variety of images, both those considered aesthetically pleasing, 
and those considered aesthetically poor . Likewise, the aesthetic 
scores for each image desirably includes a number of such scores, 
by a diverse number of professionals or laypeople, groups of which 
may be intentionally selected for their common taste (e.g., people 
who prefer The New York Times to the Wall Street Journal, people 
who read Wired Magazine, etc.). Each image is desirably scored 
manually by each professional (or each person surveyed) by 
whatever criteria the person wishes to use for deeming the 
aesthetics of the image, or according to some standard specified 
by the survey. Each image may, for example, be scored on a number 
basis, such as from zero to one-hundred, or, for example, on a 
classification basis, where there are a number of categories, such 
as "excellent," "good," "average," "poor," etc. Toyama , col. 5, 
In. 15-46. 

The Examiner takes the position that Toyama shows that selected 
examples are annotated by a user and wherein the examples provide for 
achieving a maximal disambiguation result. OA, pg . 8. The Applicants 
respectfully disagree with the Examiner. 

Claim 16 recites that the examples of multimedia content are 
selected based on at least one criterion for achieving a maximal 
disambiguation result, such that only those examples which are most 
ambiguous are selected. Toyama contains no such teaching. The only 
selection criterion discussed in Toyama is whether the training set is 
aesthetically pleasing or aesthetically poor. Thus, the Applicants 
respectfully submit that Toyama does not provide any instruction for 
selecting images based on at least one criterion for achieving a 
maximal disambiguation result, as recited in claim 16 of the present 
invent ion . 

In rejecting claims under U.S.C. §103, the Examiner bears the 
initial burden of establishing a prima facie case. In re Oetiker , 977 
F.2d 1443, 1445, 24 USPQ 1443, 1444 (Fed. Cir. 1992). To establish 
prima facie obviousness of a claimed invention, all the claim 
limitations must be taught or suggested by the prior art. In re Royka , 
490 F.2d 981, 180 USPQ 580 (CCPA 1974) . "All words in a claim must be 
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considered in judging the patentability of that claim against the prior 
art." In re Wilson , 424 F.2d 1382, 1385, 165 USPQ 494, 496 (CCPA 
1S70) . 

The Office Action also argues, "It would have been obvious, and 
desirable to have used the active selection techniques disclosed by 
Toyama in col. 5 lines 15-46 to have improved the active selection 
techniques disclosed by Toyama." The Applicants respectfully submit 
that Toyama makes no mention of an active selection technique. 
Specifically, Stubler teaches generating captions or labels for 
acquired images from preexisting captions or labels associated with 
stored images. There in no mention in Stubler of an active selection 
technique . 

The Office Action alleges that the field of endeavor is very 
similar for Stubler and Toyama since Toyama "is directed towards 
propagating annotations from a known training set to similar images 
that have do not have any associated annotations." The Applicant 
respectfully submits that Toyama does not mention or suggest 
propagating annotations from a known training set to similar images 
that do not have any associated annotations. The citation offer by the 
Examiner (col. 1, line 47 to col. 2, line 14 of Toyama) supporting, the 
Office Action's assertion is devoid of teaching relating to 
annotat ions . 

Furthermore, the Office Action fails to cite any suggestion or 
motivation in the prior art to modify the Stubler with the teachings of 
Toyama. Toyama discusses techniques for training a classifier to 
provide aesthetic scores for input images . Stubler does not 
demonstrate any appreciation of training techniques such as discussed 
in Toyama. Thus, the Applicant respectfully submits that no motivation 
or suggestion exists in the cited art to modify the Stubler with the 
teachings of Toyama. 

For at least the reasons discussed above, the Applicant 
respectfully asserts that the Office Action has not established a prima 
facie case of obviousness for claim 16. As such, the rejection of 
claim 16 should be withdrawn. Moreover, the Applicant believes that 



- 16 - 



PAGE 17733 - RCVD AT 6/27/2006 2:58:05 PM [Eastern Daylight Time] * SVR:USPTO-EFXRF-6/31 • DNIS:2738300 * CSID:71 8-544-8588 * DURATION (mm-ss):15-24 



6/27/2006 2:58 PM FROM: 718-544-8588 TO: 1-571-273-8300 PAGE: 018 OF 033 

Patent Application No. 10/056,546 

claim 16 is patentable over the cited documents and earnestly requests 
an indication of allowability for claim 16. 

Claims 17-21 

Claims 17-21 are dependent on and further limit claim 16. Since 
claim 16 is believed allowable, claims 17-21 are also believed 
allowable for at least the same reasons as claim 16 . 

Claim 25 

Claim 25 recites, "The system of claim 16, wherein the at least 
one criterion includes an ambiguity level of the selected examples." 
In rejecting claim 25, the Office Action argues that although Stubler 
does not teach wherein at least one criterion include an ambiguity 
level of selected examples, such a teaching is found at column 5, line 
15 to column 6, line 50 of Toyama . The Applicant respectfully 
disagrees with the Examiner's interpretation of Toyama. 

The Applicants find only two sentences that deal with selecting 
examples in the citation provided by the Examiner. The first sentence 
states, "For example, the images may include a set of web pages, a set 
of scanned-in pictures, a set of created pictures, a set of drawings, a 
set of page layouts, etc." The second sentence states, "The set of 
images in the training set desirably includes a wide variety of images, 
both those considered aesthetically pleasing, and those considered 
aesthetically poor." Neither sentence mentions or suggests wherein the 
at least one criterion includes an ambiguity level of the selected 
examples . 

The Examiner also argues that the statement, "Each image may, for 
example, be scored on a number basis, such as from zero to one-hundred, 
or, for example, on a classification basis, where there are a number of 
categories, such as 'excellent,' 'good,' 'average,' 'poor,' etc." found 
at column 5, lines 42-46 of Toyama teaches wherein at least one 
criterion include an ambiguity level of selected examples. OA, pg . 9. 
The Applicants, however, submit that the above-citation relates to 
aesthetic categories of images, not ambiguity levels of selected 
examples . 
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The Examiner further urges that Toyama teaches an ambiguity level 
of the selected examples at column 6, lines 19-31. This citation 
offered by the Examiner states, 

For example, Support Vector Machines build classifiers by 
identifying a hyperplane that separates a set of positive and 
negative examples with a maximum margin. In the linear form of SVM 
that is employed in one embodiment, the margin is defined by the 
distance of the hyperplane to the nearest positive and negative 
cases for each class. Maximizing the margin can be express as an 
optimization problem and search and optimization thus lay at the 
core of different SVM-based training methods. A post -process ing 
procedure described in the Piatt reference is used that employs 
regularized maximum likelihood fitting to produce estimations of 
posterior probabilities. The method fits a sigmoid to the score 
that is output by the SVM classifier. Toyama, col. 6, In. 19-31. 

The Applicants respectfully submit that this passage relates to 
using Support Vector Machines (see 

http : //en . wikipedia . org/wiki/Support_vector_machines attached herewith) 
for creating functions from a set of labeled training data. The 
passage contains no teaching or suggestion of using ambiguity levels of 
selected examples. 

For at least these reasons, and the reasons given for claim 16, 
claim 25 is believed allowable. The Applicant therefore earnestly 
requests an indication of allowability for claim 25. 

Claim 26 

Claim 26 recites, w The system of claim 16, wherein the at least 
one criterion includes a confidence level of the selected examples, the 
confidence level being inversely proportional to a distance of a new 
feature of the selected examples from a separating hyperplane in an 
induced higher dimensional feature space." In rejecting claim 26, the 
Office Action argues that although Stubler does not teach wherein the 
at least one criterion includes a confidence level of the selected 
examples, the confidence level being inversely proportional to a 
distance of a new feature of the selected examples from a separating 
hyperplane in an induced higher dimensional feature space, such a 
teaching is found at column 5, line 15 to column 6, line 50 of Toyama. 
The Applicant respectfully disagrees with the Examiner's interpretation 
of Toyama . 
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The Applicants find only two sentences that deal with selecting 
examples in the citation provided by the Examiner. The first sentence 
states, "For example, the images may include a set of web pages, a set 
of scanned-in pictures, a set of created pictures, a set of drawings, a 
set of page layouts, etc." The second sentence states, "The set of 
images in the training set desirably includes a wide variety of images, 
both those considered aesthetically pleasing, and those considered 
aesthetically poor." Neither sentence mentions or suggests wherein the 
at least one criterion includes a confidence level of the selected 
examples, the confidence level being inversely proportional to a 
distance of a new feature of the selected examples from a separating 
hyperplane in an induced higher dimensional feature space. 

The Examiner also argues that the statement, "Each image may, for 
example, be scored on a number basis, such as from zero to one-hundred, 
or, for example, on a classification basis, where there are a number of 
categories, such as "excellent,' "good, ' "average,' "poor,' etc." found 
at column 5, lines 42-46 of Toyama teaches wherein the at least one 
criterion includes a confidence level of the selected examples, the 
confidence level being inversely proportional to a distance of a new 
feature of the selected examples from a separating hyperplane in an 
induced higher dimensional feature space. OA, pg . 9. The Applicants, 
however, submit that the above-citation relates to aesthetic categories 
of images, not a confidence level inversely proportional to a distance 
of a new feature of the selected examples from a separating hyperplane 
in an induced higher dimensional feature space. 

The Examiner further urges that Toyama teaches a confidence level 
inversely proportional to a distance of a new feature of the selected 
examples from a separating hyperplane in an induced higher dimensional 
feature space at column 6, lines 19-31. This citation offered by the 
Examiner states, 

For example, Support Vector Machines build classifiers by 
identifying a hyperplane that separates a set of positive and 
negative examples with a maximum margin. In the linear form of SVM 
that is employed in one embodiment, the margin is defined by the 
distance of the hyperplane to the nearest positive and negative 
cases for each class. Maximizing the margin can be express as an 
optimization problem and search and optimization thus lay at the 
core of different SVM-based training methods. A pos t -proces s ing 
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procedure described in the Piatt reference is used that employs 
regularized maximum likelihood fitting to produce estimations of 
posterior probabilities. The method fits a sigmoid to the score 
that is output by the SVM classifier. Toyama , col. 6, In. 19-31. 

The Applicants respectfully submit that this passage relates to 
using Support Vector Machines (see 

http : //en . wikipedia . org/wiki /Support _vector_machine s attached herewith) 
for creating functions from a set of labeled training data. The 
passage contains no teaching or suggestion of using a confidence level 
inversely proportional to a distance of a new feature of the selected 
examples from a separating hyperplane in an induced higher dimensional 
feature space. 

For at least these reasons, and the reasons given for claim 16, 
claim 26 is believed allowable. The Applicant therefore earnestly 
requests an indication of allowability for claim 26. 

Claim 22 

Claim 22 recites, in part, "selecting of examples of multimedia 
content to be annotated by a user, wherein the examples of multimedia 
content are selected based on at least one criterion for achieving a 
maximal disambiguation result such that only those examples which are 
most ambiguous are selected . " 

In rejecting claim 22, the Office Action argues that although 
Stubler does not teach actively selecting examples of multimedia 
content to be annotated by a user, wherein the examples of multimedia 
content are selected based on at least one criterion for achieving a 
maximal disambiguation result, this teaching is found in Toyama. The 
Office Action cites column 5, lines 15-46 of Toyama in support of its 
position . 

The cited passages of Toyama state, 

In this section of the detailed description, a method for training 
a classifier, according to one embodiment of the invention, is 
described. Referring to FIG. 2, a flowchart of the method is 
shown. In 200, a training set of images and corresponding 
aesthetic scores for the images is input . The images are desirably 
the same type of images that are to be later judged. For example, 
the images may include a set of web pages, a set of scanned-in 
pictures, a set of created pictures, a set of drawings, a set of 
page layouts, etc. The aesthetic scores for the images are 
desirably made by graphics professionals. Thus, the web pages are 
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desirably scored by professional web page designers, the scanned- 
in pictures by professional photographers, the created pictures by 
professional artists, the drawings by professional drawers, etc. 

The set of images in the training set desirably includes a wide 
variety of images, both those considered aesthetically pleasing, 
and those considered aesthetically poor . Likewise, the aesthetic 
scores for each image desirably includes a number of such scores, 
by a diverse number of professionals or laypeople, groups of which 
may be intentionally selected for their common taste (e.g., people 
who prefer The New York Times to the Wall Street Journal, people 
who read Wired Magazine, etc.). Each image is desirably scored 
manually by each professional (or each person surveyed) by 
whatever criteria the person wishes to use for deeming the 
aesthetics of the image, or according to some standard specified 
by the survey. Each image may, for example, be scored on a number 
basis, such as from zero to one-hundred, or, for example, on a 
classification basis, where there are a number of categories, such 
as "excellent," "good," "average," "poor," etc. Toyama , col. 5, 
In. 15-46. 

The Examiner takes the position that Toyama shows that selected 
examples are annotated by a user and wherein the examples provide for 
achieving a maximal disambiguation result. OA, pg . 8. The Applicants 
respectfully disagree with the Examiner. 

Claim 22 recites that the examples of multimedia content are 
selected based on at least one criterion for achieving a maximal 
disambiguation result, such that only those examples which are most 
ambiguous are selected. Toyama contains no such teaching. The only 
selection criterion discussed in Toyama is whether the training set is 
aesthetically pleasing or aesthetically poor. Thus, the Applicants 
respectfully submit that Toyama does not provide any instruction for 
selecting images based on at least one criterion for achieving a 
maximal disambiguation result, as recited in claim 22 of the present 
invention . 

In rejecting claims under U.S.C. §103, the Examiner bears the 
initial burden of establishing a prima facie case. In re Oetiker , 977 
F.2d 1443, 1445, 24 USPQ 1443, 1444 (Fed. Cir. 1992). To establish 
prima facie obviousness of a claimed invention, all the claim 
limitations must be taught or suggested by the prior art. In re Royka , 
490 F.2d 981, 180 USPQ 580 (CCPA 1974). "All words in a claim must be 
considered in judging the patentability of that claim against the prior 
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art." In re Wilson , 424 F.2d 1382, 1385, 165 USPQ 494, 496 (CCPA 
1970) . 

The Office Action also argues, "It would have been obvious and 
desirable to have used the active selection techniques disclosed by 
Toyama in col. 5 lines 15-46 to have improved the active selection 
techniques disclosed by Toyama." The Applicants respectfully submit 
that Toyama makes no mention of an active selection technique. 
Specifically, Stubler teaches generating captions or labels for 
acquired images from preexisting captions or labels associated with 
stored images. There in no mention in Stubler of an active selection 
technique. 

The Office Action alleges that the field of endeavor is very 
similar for Stubler and Toyama since Toyama "is directed towards 
propagating annotations from a known training set to similar images 
that have do not have any associated annotations." The Applicant 
respectfully submits that Toyama does not mention or suggest 
propagating annotations from a known training set to similar images 
that do not have any associated annotations. The citation offer by the 
Examiner (col. 1, line 47 to col. 2, line 14 of Toyama) supporting the 
Office Action's assertion is devoid of teaching relating to 
annotations . 

Furthermore, the Office Action fails to cite any suggestion or 
motivation in the prior art to modify the Stubler with the teachings of 
Toyama. Toyama discusses techniques for training a classifier to 
provide aesthetic scores for input images. Stubler does not 
demonstrate any appreciation of training techniques such as discussed 
in Toyama. Thus, the Applicant respectfully submits that no motivation 
or suggestion exists in the cited art to modify the Stubler with the 
teachings of Toyama. 

For at least the reasons discussed above, the Applicant 
respectfully asserts that the Office Action has not established a prima 
facie case of obviousness for claim 22. As such, the rejection of 
claim 22 should be withdrawn. Moreover, the Applicant believes that 
claim 22 is patentable over the cited documents and earnestly requests 
an indication of allowability for claim 22. 
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Claim 27 

Claim 27 recites, "The computer program product of claim 22, 
wherein the at least one criterion includes an ambiguity level of the 
selected examples." In rejecting claim 27, the Office Action argues 
that although Stubler does not teach wherein at least one criterion 
include an ambiguity level of selected examples, such a teaching is 
found at column 5, line 15 to column 6, line 50 of Toyama . The 
Applicant respectfully disagrees with the Examiner's interpretation of 
Toyama . 

The Applicants find only two sentences that deal with selecting 
examples in the citation provided by the Examiner. The first sentence 
states, "For example, the images may include a set of web pages, a set 
of scanned-in pictures, a set of created pictures, a set of drawings, a 
set of page layouts, etc." The second sentence states, "The set of 
images in the training set desirably includes a wide variety of images, 
both those considered aesthetically pleasing, and those considered 
aesthetically poor." Neither sentence mentions or suggests wherein the 
at least one criterion includes an ambiguity level of the selected 
examples . 

The Examiner also argues that the statement, "Each image may, for 
example, be scored on a number basis, such as from zero to one-hundred, 
or, for example, on a classification basis, where there are a number of 
categories, such as * excellent,' 'good,' 'average,' 'poor, ' etc." found 
at column 5, lines 42-46 of Toyama teaches wherein at least one 
criterion include an ambiguity level of selected examples. OA, pg . 9. 
The Applicants, however, submit that the above-citation relates to 
aesthetic categories of images, not ambiguity levels of selected 
examples . 

The Examiner further urges that Toyama teaches an ambiguity level 
of the selected examples at column 6, lines 19-31. This citation 
offered by the Examiner states, 

For example, Support Vector Machines build classifiers by 
identifying a hyperplane that separates a set of positive and 
negative examples with a maximum margin. In the linear form of SVM 
that is employed in one embodiment, the margin is defined by the 
distance of the hyperplane to the nearest positive and negative 
cases for each class. Maximizing the margin can be express as an 
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optimization problem and search and optimization thus lay at the 
core of different SVM-based training methods. A post -processing 
procedure described in the Piatt reference is used that employs 
regularized maximum likelihood fitting to produce estimations of 
posterior probabilities. The method fits a sigmoid to the score 
that is output by the SVM classifier. Toyama , col. 6, In. 19-31. 

The Applicants respectfully submit that this passage relates to 
using Support Vector Machines (see 

ht tp : / /en . wikipedia . org/ wiki /Support_vector_machines attached herewith) 
for creating functions from a set of labeled training data. The 
passage contains no teaching or suggestion of using ambiguity levels of 
selected examples. 

For at least these reasons, and the reasons given for claim 22, 
claim 27 is believed allowable. The Applicant therefore earnestly 
requests an indication of allowability for claim 27. 

Claim 28 

Claim 28 recites, "The computer program product of claim 22, 
wherein the at least one criterion includes a confidence level of the 
selected examples, the confidence level being inversely proportional to 
a distance of a new feature of the selected examples from a separating 
hyperplane in an induced higher dimensional feature space." In 
rejecting claim 28, the Office Action argues that although Stubler does 
not teach wherein the at least one criterion includes a confidence 
level of the selected examples, the confidence level being inversely 
proportional to a distance of a new feature of the selected examples 
from a separating hyperplane in an induced higher dimensional feature 
space, such a teaching is found at column 5, line 15 to column 6, line 
50 of Toyama. The Applicant respectfully disagrees with the Examiner's 
interpretation of Toyama. 

The Applicants find only two sentences that deal with selecting 
examples in the citation provided by the Examiner. The first sentence 
states, "For example, the images may include a set of web pages, a set 
of scanned-in pictures, a set of created pictures, a set of drawings, a 
set of page layouts, etc." The second sentence states, w The set of 
images in the training set desirably includes a wide variety of images, 
both those considered aesthetically pleasing, and those considered 
aesthetically poor." Neither sentence mentions or suggests wherein the 
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at least one criterion includes a confidence level of the selected 
examples, the confidence level being inversely proportional to a 
distance of a new feature of the selected examples from a separating 
hyperplane in an induced higher dimensional feature space. 

The Examiner also argues that the statement, "Each image may, for 
example, be scored on a number basis, such as from zero to one-hundred, 
or, for example, on a classification basis, where there are a number of 
categories, such as 'excellent, ' 'good, ' 'average,' 'poor,' etc." found 
at column 5, lines 42-46 of Toyama teaches wherein the at least one 
criterion includes a confidence level of the selected examples, the 
confidence level being inversely proportional to a distance of a new 
feature of the selected examples from a separating hyperplane in an 
induced higher dimensional feature space. OA, pg. 9. The Applicants, 
however, submit that the above-citation relates to aesthetic categories 
of images, not a confidence level inversely proportional to a distance 
of a new feature of the selected examples from a separating hyperplane 
in an induced higher dimensional feature space. 

The Examiner further urges that Toyama teaches a confidence level 
inversely proportional to a distance of a new feature of the selected 
examples from a separating hyperplane in an induced higher dimensional 
feature space at column 6, lines 19-31. This citation offered by the 
Examiner states, 

For example, Support Vector Machines build classifiers by 
identifying a hyperplane that separates a set of positive and 
negative examples with a maximum margin. In the linear form of SVM 
that is employed in one embodiment, the margin is defined by the 
distance of the hyperplane to the nearest positive and negative 
cases for each class. Maximizing the margin can be express as an 
optimization problem and search and optimization thus lay at the 
core of different SVM-based training methods. A pos t -proces s ing 
procedure described in the Piatt reference is used that employs 
regularized maximum likelihood fitting to produce estimations of 
posterior probabilities. The method fits a sigmoid to the score 
that is output by the SVM classifier. Toyama, col. 6, In. 19-31. 

The Applicants respectfully submit that this passage relates to 
using Support Vector Machines (see 

http : //en . wikipedia . org / wiki / Support _vector__machines attached herewith) 
for creating functions from a set of labeled training data. The 
passage contains no teaching or suggestion of using a confidence level 

- 25 - 

PAGE 26/33 * RCVD AT 6/27/2006 2:58:05 PM [Eastern Daylight Time] * SVR:USPTO-EFXRF^/31 * DNIS:2738300 * CSID: 71 8-544-8588 * DURATION <mm-ss):15-24 



6/27/2006 2:58 PM FROM: 718-544-8588 TO: 1-571-273-8300 PAGE: 027 OF 033 



Patent Application No. 10/056,546 



inversely proportional to a distance of a new feature of the selected 
examples from a separating hyperplane in an induced higher dimensional 
feature space. 

For at least these reasons, and the reasons given for claim 22, 
claim 28 is believed allowable. The Applicant therefore earnestly 
requests an indication of allowability for claim 28. 

CONCLUSION 

In view of the forgoing remarks, it is respectfully submitted that 
this case is now in condition for allowance and such action is 
respectfully requested. If any points remain at issue that the 
Examiner feels could best be resolved by a telephone interview, the 
Examiner is urged to contact the attorney below. 

No fee is believed due with this Amendment, however, should a fee 
be required please charge Deposit Account 50-0510. Should any 
extensions of time be required, please consider this a petition thereof 
and charge Deposit Account 50-0510 the required fee. 

Please charge Deposit Account 50-0510 the amount of $120 for a 
one-month extension for the period for response to the Office Action. 
No other fee is believed due with this Amendment, however, should a fee 
be required please charge Deposit Account 50-0510. Should any 
additional extensions of time be required, please consider this a 
petition thereof and charge Deposit Account 50-0510 the required fee. 



Dated: June 27, 2006 



Respectfully submitted, 





Cdo Tuchman, Reg. No. 4 5,92 4 
Law Office of Ido Tuchman 
82-70 Beverly Road 
Kew Gardens, NY 11415 
Telephone (718) 544-1110 
Facsimile (718) 544-8588 
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